Log structured block device for hard disk drive

ABSTRACT

Example apparatus and methods provide a log structured block device for a hard disk drive (HDD). Data that is to be stored on an HDD is serialized and written as a series of data blocks using a sequential write. Information about where individual data blocks were supposed to be written (e.g., actual address, neighboring data blocks), where data blocks were actually written, and how often data blocks are accessed is maintained. During garbage collection, data blocks that are being accessed with similar frequencies may be relocated together, with the most frequently accessed (e.g., hottest) data blocks migrating to the outer cylinders of the disk and the least frequently accessed (e.g., coldest) data blocks migrating to the inner cylinders. Blocks stored in the same temperature regions that were intended to be located together when written may be repositioned to facilitate sequential reads.

REFERENCE TO RELATED APPLICATION

This Application is a Divisional of U.S. application Ser. No. 14/728,038filed on Jun. 2, 2015, the contents of which are hereby incorporated byreference in their entirety.

BACKGROUND

A hard disk drive (HDD) stores data on one or more platters that have aplurality of concentric cylinders. The platters spin to move differentlocations under a read/write head(s). The data stored by the HDD may beorganized in files, objects, records, tables, byte-streams, or in otherways. In the early days of computing, an HDD may have stored data for asingle computer running a single process. When data was written in thisscenario, there was only a single place from which the data would beprovided and only a single HDD to which the data would be written. Whendata was going to be read, there was only one HDD from which the datacould be read and only one place to which the data would be provided.Since there was only a single source and a single destination, attentioncould be paid to optimizing the data for write efficiency, for readefficiency, or for balancing write and read efficiency.

As computing progressed, HDDs began to store data for multiple processesassociated with a computer. Decisions concerning where to store anyparticular data had to be made, either explicitly when the data wasstored, or implicitly by configuring a program or utility. Decisionsalso had to be made concerning how to write data that arrived fromdifferent processes at different times. Similarly, decisions concerningto where data was to be provided had to be made. File systems andoperating systems facilitated virtualizing the storage devices andimproved some of the decision making.

As computing progressed even further, computers may have begun tointeract on the storage side with networks of HDDs and on the processingside with networks of processes or computers. In this scenario, thelocations available to store data expanded exponentially and the numberof processes seeking to store or retrieve data also expandedexponentially. Still, decisions concerning where or how any particularpiece of data was to be stored may have been made on an as-needed basisusing heuristics that were designed in a single-disk/single-process era.This new problem is unique to data storage systems that interact withmultiple data providers. It is not the type of problem that has existedfor a long time or for which a paper and pencil solution exists sincethe writing and positioning of electronic data is involved.

For many applications, HDD performance may be a significant concern. HDDwrite performance may be affected by how data is written (e.g.,randomly, sequentially) and HDD read performance may be affected by howdata is read (e.g., randomly, sequentially). Traditionally, HDDs providehigher performance when accessed sequentially instead of randomly.However, the ability to perform random access is one of the criteria bywhich HDD are selected over other storage devices (e.g., tape drives)for a system. Conventional file systems, operating systems,applications, device drivers, or other processes that may access an HDDmay be aware of the sequential versus random performance concern.However, interactions between ever increasing layers of virtualizationwith ever increasing complexity may frustrate attempts to organize datafor sequential access.

Advances in HDDs have produced increases in capacity. However, theincreases in capacity have not been matched with equal increases in HDDperformance, even while storage system designers may equate capacitywith performance. The disconnect between capacity and performance mayfrustrate systems designers. Additionally, the disconnect may lead toinefficiencies in data storage, which in turn may lead to over-spendingand excessive energy consumption. The disconnect between capacity andstorage caused by conventional storage heuristics may cause aconventional HDD to have excess capacity at a given performance level.In one extreme example, an HDD may be “short stroked”, where the fastestouter cylinders are used to store data to ensure the best possibletransfer rate and the shortest seek latency. While the fastest outercylinders may be over-utilized, middle and especially inner cylindersmay be under-utilized, if even used at all.

Conventional systems may have sought to optimize write performance byusing sequential writes, even for data from multiple processes. Theseconventional systems may have used log structured input/output (i/o) inattempts to produce more sequential writes. In log structured i/o, datais written to a device sequentially, in the order that the data arrives,regardless of a location intended for the data by the entity thatprovided the data to the log structured device. Thus, data from onesource may be interleaved with data from other sources and the combinedset of data written sequentially. Files may not be written contiguously.A log structured i/o device may maintain a block map that tracks wherethe data is actually written and where the entity that provided the datawanted the data to be written. Conventional log structured i/o tends tooptimize writing over reading, which may lead to significant performancedegradation when the data that was sequentially written needs to be readback in.

Log structuring may be implemented at the file system level. The filesystem may be responsible for assigning blocks to the data in the filesmanaged by the file system. A file system may aggregate writes to filesso that the writes are sequential and therefore faster than randomwrites. While the aggregation that leads to sequential writes mayimprove write speed, reads of a file that might be sequential in anotherfile system may or may not be sequential in a log structured filesystem. For example, a file that might have been written contiguously inanother system may not have been written contiguously in a logstructured i/o device. Whether the read is sequential will depend onwhat else, if anything, was written at the same time as the file to beread. If the file to be read was the only thing written at the point intime when the file was written, then the file may have been writtensequentially. However, if other data was written at the same time as thefile to be read, then other data blocks may have been interspersed withthe file to be read, which may in turn lead to non-sequential reads forthe file. As HDDs store data for more and more processes associated withmore and more computers, the likelihood that any single file will bewritten in isolation becomes less and less. Thus, HDD read performancemay continue to suffer accordingly.

Log structuring has been employed in conventional solid state drives(SSD). Log structuring may be implemented at the block level inFLASH-based SSDs because this type of device is structured in pages thatneed to be erased before they can be rewritten. Log structuring is wellsuited to this type of device because writes are sequential and there islittle to no penalty for non-sequential reads. When implemented in anSSD, log structuring may be referred to as a “FLASH translation layer.”The block translation map produced by log structuring may be stored onthe SSD.

While log structuring may provide improved write performance for SSDswithout the corresponding read performance degradation found in HDDs,the log structuring approach for SSDs may be compromised by SSD garbagecollection. An issue may occur during garbage collection because livedata in a page on an SSD may need to be moved before the page can bereclaimed. Thus, garbage collection for SSD may constrain long-termwrite throughput of FLASH-based SSD. This garbage collection issue maybe associated with the observed phenomenon that while SSDs may be veryfast early in their deployment, they experience gradual degradation inperformance after deployment.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate various example systems, methods,and other example embodiments of various aspects of the invention. Itwill be appreciated that the illustrated element boundaries (e.g.,boxes, groups of boxes, or other shapes) in the figures represent oneexample of the boundaries. One of ordinary skill in the art willappreciate that in some examples one element may be designed as multipleelements or that multiple elements may be designed as one element. Insome examples, an element shown as an internal component of anotherelement may be implemented as an external component and vice versa.Furthermore, elements may not be drawn to scale.

Prior Art FIG. 1 illustrates data being provided to a hard disk drive(HDD).

Prior Art FIG. 2 illustrates data being provided to an HDD.

FIG. 3 illustrates data being provided to an HDD.

FIG. 4 illustrates data being processed by a log structured blockdevice.

FIG. 5 illustrates different cylinder zones on a platter of an HDD.

FIG. 6 illustrates an example method associated with improving HDDperformance and efficiency using a log structured block device thatperforms selective data relocation.

FIG. 7 illustrates an example method associated with improving HDDperformance and efficiency using a log structured block device thatperforms selective data relocation.

FIG. 8 illustrates an example apparatus that improves HDD performanceand efficiency using a log structured block device that performsselective data relocation.

DETAILED DESCRIPTION

Example apparatus and methods improve performance and efficiency for HDDsystems. Example apparatus and methods provide a log structured blockdevice that produces the efficiencies associated with sequential writesto an HDD while mitigating the issues of non-sequential readsexperienced by conventional systems. An example log structured blockdevice may serialize write operations, regardless of the source of thedata to be written. Serialize, as used herein, means to arrange datablocks in an order. In one embodiment, serializing items includesarranging them in the order in which they were received. For example,data from different sources may be combined into a data set in the orderin which they were received and then the entire data set may be writtensequentially. An example log structured block device may alsoselectively reorganize data that was written to the HDD to improve oroptimize read operations by increasing the likelihood of sequentialreads. The selective reorganization uses a block map that is maintainedduring writing and relocation. The block map stores information aboutwhere a provider of data to be written wanted the data to be written andwhere the log structured block device actually stored the data. Theinformation about where the provider wanted the data to be written mayidentify data blocks that were intended to be written togethersequentially (e.g., blocks from a file, blocks from a data set). Theinformation about where the provider wanted the data to be written mayalso identify physical block addresses.

The selective reorganization of data that was written to an HDD usinglog structured i/o may occur, for example, during backgrounddefragmentation of a device. The selective reorganization may be based,at least in part, on a heat map that tracks how often certain data isread or the frequency with which certain data is read. The term “heatmap” refers to data or a data structure that tracks accesses to datawritten on the HDD. In one embodiment, when a block is touched (e.g.,read, accessed) a value in the heat map is incremented. Accesses maycause a value for a block to be increased. Values may be decreasedperiodically (e.g., once per hour, once per day), in response to anevent, in response to a signal, or at other times. In addition to beingcollocated to increase the likelihood of sequential reads, data that ismore frequently read may be migrated to faster cylinders on a spinningplatter while data that is less frequently read may be migrated toslower cylinders on the spinning platter.

Thus, example apparatus and methods improve over conventional systems byoptimizing for write performance at the time when data is written butthen, over time, based on usage, re-optimizing for read performance. Inone embodiment, a log structured block device is implemented using anextrinsic virtualization layer. Relocating data to improve readperformance may be performed as part of a garbage collection or otherselective relocation process.

Over time, due to data being trimmed, overwritten, or otherwisemanipulated, parts of a log may become porous. Areas that are porous mayhave little live data and may have that live data interspersed withinactive data or empty locations. Log structured writes are performedsequentially and thus benefit from having large areas into which longsequential writes can be performed. Thus, data located in porous areasmay be relocated so that the porous area becomes more suitable forsequential writes associated with a log structured i/o approach.

Conventional garbage collection may employ unsophisticated or even naïveheuristics to identify areas in which garbage collection may beperformed. For example, a naïve approach may identify sufficiently largeregions whose porosity indicates that garbage collection is appropriate.Data in these regions may be moved to the end of the log resulting innew areas that can be written sequentially by the log structured i/odevice.

Some file systems may improve on this naïve garbage collection for logstructured devices by applying a heuristic that related files or relatedblocks should be kept together. This heuristic may be based on theassumption that files in a single directory may be more likely to beaccessed together than files from different directories and that data ina single file is likely to be read together. These types of file systemsmay bias the placement of live data that is moved during garbagecollection so that data from the same file will be located together andso that files from the same directory are located together. Ideally,data from the same file would be positioned to facilitate sequentialreads.

Example apparatus and methods employ more sophisticated heuristics toguide or control the relocation of live data during garbage collectionor other relocations. Example apparatus and methods maintain a heat mapfor blocks stored on an HDD. The term “heat map” refers to a datastructure or collection of data that stores information about access(e.g., read) activity for a block. Blocks that are read more often thanother blocks may be referred to as “hot” blocks while blocks that areread less often than other blocks may be referred to as “cold” blocks. Ablock on an HDD is assigned a value that depends, at least in part, onhow many times a block has been read, how frequently the block is read,or other accesses to the data already written to the HDD. In oneembodiment, each time the block is read the value may be incremented orincreased. The value for a block may also be decremented or reduced. Forexample, values for blocks may be reduced periodically, in response to asignal, in response to an event, or at other times.

When garbage collection occurs for an HDD, the position to which livedata is relocated on the HDD may depend on the value in the heat map fora block. For example, data blocks with the highest values may becollocated in one area (e.g., first set of cylinders) to increase thelikelihood of sequential reads of that data while data blocks with thelowest values may also be collocated to another area (e.g., differentset of cylinders) to increase the likelihood of sequential reads of thatdata. In one embodiment, values for the heat map may be binned and datablocks associated with the different bins may be collocated. In oneembodiment, the most frequently read data may be placed on the outerrings of the HDD. In one embodiment, data may be positioned from theouter rings to the inner rings in direct relation to the values in theheat map.

Once blocks that have similar heat map values have been identified, andonce destination cylinders have been identified for blocks with thosesimilar heat map values, additional refinements to the ultimate locationfor blocks to be relocated may be made. One additional refinementrelocates data blocks having similar heat map values based, at least inpart, on information from the log concerning the intended location forthe block. The intended location for the block may be identified, forexample, by the provider of the block. Recall that the block map storesthe location intended by the provider of the block and the actuallocation used by the log structured block device. When data is movedduring garbage collection, the actual location at which the block islocated is updated but the original intended location may remain thesame.

Over time, as garbage collection proceeds or is repeated, data will berepositioned with a first granularity based on the number of reads, readfrequency, or other access data. This may increase the likelihood ofsequential reads for the relocated data. For data that is read withsimilar frequencies, a finer granularity may be achieved by positioningtogether data that was intended to be positioned together. For example,data blocks from the same file or same data set may be positionedtogether. This may further improve the likelihood of sequential readsand thus improve HDD efficiency and performance.

Turning now to FIG. 1, data 100 is received at apparatus 110. Apparatus110 may be, for example, an HDD. Data 100 may be associated with a filesystem 130 or a process 120. In the beginning, data 100 may have beenreceived from a single location (e.g., single process) and would bewritten to a single location (e.g., apparatus 110). Optimizing thewriting of data 100 to produce sequential writes was straightforward, aswas optimizing the writing of data 100 to produce sequential reads. Thesingle provider of data may have been able to easily control the mannerin which the data was stored to optimize for either writing or reading.

FIG. 2 illustrates data 200 being received at apparatus 210. Apparatus210 may be, for example, an HDD. Data 200 may be received from multiplelocations but is still stored at a single location. The multiplelocations may include multiple processes, for example process 220through process 222 and process 224 through process 226. The multiplelocations may also include multiple file systems, for example filesystems 230 and 232. The multiple locations produce different data atdifferent times and thus data 200 may include a combination of data fromthe various sources. If apparatus 210 is a log structured device, thenit may be straightforward to optimize apparatus 210 for sequentialwrites. However, conventional log structured devices may experience theundesired result of more frequent random reads as a result of thetime-of-arrival based serialization of received data.

FIG. 3 illustrates data 300 being received at apparatus 310 andapparatus 312. Apparatus 310 and 312 may be, for example, hard diskdrives. Data 300 is received from multiple locations and may be storedat multiple locations. For example, data 300 may be received from filesystems 330 through 332 and from processes 320 through 322 and processes324 through 326. It may have been possible to use a log structureddevice to produce sequential writes at the various HDD to which the data300 is written. But the problems described above may have produced HDDperformance that was significantly sub-optimal due, at least in part, tothe serialization of data from multiple devices, file systems,processes, or other providers. The problems may be exacerbated when datathat was intended to be written together is distributed onto differentapparatus. For example, data that was intended to be written togethermay be stored partially on apparatus 310 and partially on apparatus 312.Example apparatus and methods facilitate improving the situationdepicted in any of FIGS. 1-3. Example apparatus and methods provide alog structured block device that initially optimizes for sequentialwrites but then uses a heat map based approach to relocate data during,for example, garbage collection, to optimize for sequential reads.

The problem illustrated in FIG. 3 continues to expand as more and moreprocesses store more and more data to more and more devices. In anenvironment like that depicted in FIG. 3, devices may face anunpredictable or random workload that compromises performance. Deviceutilization may be near 100%, but device performance may be well underthat level. The problem illustrated in FIG. 3 may be exacerbated bycertain types of drives (e.g., shingled magnetic recording (SMR))drives. Thus, in one embodiment, the log structured block device maycontrol data placement and relocation for an SMR drive.

FIG. 4 illustrates data 400 being processed by a log structured blockdevice 410 that maintains a heat map 420 and a log 430. Apparatus 440may be, for example, an HDD or a shingled magnetic recording (SMR)drive. Log structured block device 410 optimizes writes of data 400 toapparatus 440 and then, over time, relocates data on apparatus 440 toimprove HDD read performance and efficiency. Log structured block device410 may, regardless of which file system or process provided blocks indata 400, write the blocks sequentially. The blocks may be writtensequentially in the order they were received or may be subjected to somepre-processing or re-ordering in, for example, a write buffer. Logstructured block device 410 maintains a log 430 of where data blockswere intended to be written and then where they were actually written.Log structured block device 410 also maintains a heat map 420 thattracks, for example, how frequently a block on apparatus 440 is accessed(e.g., read). Entries in the heat map 420 may be manipulated a first way(e.g., increased) when a block on apparatus 440 is accessed and may bemanipulated another way (e.g., decreased) at different times underdifferent conditions. For example, entries may be decreasedperiodically, upon the receipt of a signal, upon the occurrence of anevent (e.g., new write of other data, new read of other data), or atother times.

When apparatus 440 is subject to garbage collection or other actionsthat will involve data relocation, log structured block device 410 maytake the opportunity to relocate data to improve read performance. Forexample, data that is being read with similar frequencies or in similarnumbers may be moved to similar regions of apparatus 400. Additionally,data that ends up in similar regions of apparatus 400 may then berelocated to enhance the possibility of sequential reads. In oneembodiment, data that was intended to be written together (e.g., datafrom same file, data from same data set) can be relocated to actually betogether (e.g., contiguous with respect to sequential reading). Theintention that data was intended to be written together can be discernedfrom the log of intended write locations. In one embodiment, data thatis observed to be read together is placed in locations that willincrease the likelihood of sequential reads. In one embodiment, bothintended locations and observed behavior may be considered. In oneembodiment the observed behavior may trump the intended write locationsand data that is actually read together may be placed togetherregardless of the originally intended write locations.

In one embodiment, to further improve performance and efficiency, datathat is read most frequently or the most times will be put on outercylinders of apparatus 400 while data that is read least frequently or afewer number of times will be put on inner cylinders of apparatus 400.This may cause cylinders that are “in the middle” (e.g., neitheroutermost rings nor innermost rings) to be available for sequentialwrites by the log structured block device 400.

While a single apparatus 440 is illustrated, log structured block device410 may store data on multiple HDD. Thus, in one embodiment, therelocation may include moving data between HDD. While log structuredblock device 410 is illustrated separate from apparatus 440, in oneembodiment, log structured block device 410 may be incorporated intoapparatus 440. For example, an ASIC, FPGA, or other circuit may be addedto apparatus 440.

FIG. 5 illustrates a spinning disk. The spinning disk includes anoutermost region 500, an innermost region 510, and an in-between area520. Data can be read most efficiently from outermost region 500 andleast efficiently from innermost region 510. Data in the in-between area520 is read less efficiently than area 500 but more efficiently thanarea 510. Example apparatus and methods may relocate data fromin-between area 520 to area 500 or area 510 leaving appropriate sizedareas to perform sequential writes in area 520.

Some portions of the detailed descriptions herein are presented in termsof algorithms and symbolic representations of operations on data bitswithin a memory. These algorithmic descriptions and representations areused by those skilled in the art to convey the substance of their workto others. An algorithm, here and generally, is conceived to be asequence of operations that produce a result. The operations may includephysical manipulations of physical quantities. Usually, though notnecessarily, the physical quantities take the form of electrical ormagnetic signals capable of being stored, transferred, combined,compared, and otherwise manipulated. The physical manipulations create aconcrete, tangible, useful, real-world result.

It has proven convenient at times, principally for reasons of commonusage, to refer to these signals as bits, values, elements, symbols,characters, terms, or numbers. It should be borne in mind, however, thatthese and similar terms are to be associated with the appropriatephysical quantities and are merely convenient labels applied to thesequantities. Unless specifically stated otherwise, it is to beappreciated that throughout the description, terms including processing,computing, and determining refer to actions and processes of a computersystem, logic, processor, or similar electronic device that manipulatesand transforms data represented as physical (electronic) quantities.

Example methods may be better appreciated with reference to flowdiagrams. For purposes of simplicity of explanation, the illustratedmethodologies are shown and described as a series of blocks. However, itis to be appreciated that the methodologies are not limited by the orderof the blocks, as some blocks can occur in different orders orconcurrently with other blocks from that shown and described. Moreover,less than all the illustrated blocks may be required to implement anexample methodology. Blocks may be combined or separated into multiplecomponents. Furthermore, additional or alternative methodologies canemploy additional, not illustrated blocks.

FIG. 6 illustrates a method 600 associated with improving HDDperformance and efficiency using a log structured block device thatperforms selective data relocation. Method 600 includes, at 610,receiving data to store on a hard disk drive (HDD). The HDD has at leastone spinning platter. The spinning platter is configured with aplurality of concentric writeable cylinders. The HDD may be, forexample, a shingled magnetic recording (SMR) device. The data may bereceived from one process, one file system or one device, or may bereceived from a plurality of processes, a plurality of file systems, ora plurality of devices.

Method 600 also includes, at 620, serializing the data to store.Serializing the received data into serialized data may include, forexample, arranging the received data in the order in which it wasreceived. As described above, when there are multiple providers of data,data that ought to be stored together (e.g., data from the same file)may be received at the same time as other data which may cause data thatwas intended to be stored together to be interspersed with other data.In one embodiment, serializing the data may therefore include moving adata block out of the order in which it was received and into an orderdetermined by the presence of a related data block. For example, datathat ought to be stored together may be recognized and data that wasinterspersed with this data that ought to be stored together may bemoved out of the order in which it was received. The amount ofre-organization that is performed before doing a sequential write maydepend on the size of a write buffer available to method 600.

Method 600 also includes, at 630, writing the serialized data onto theHDD as a series of data blocks using a sequential write operation.

Method 600 also includes, at 640, maintaining a block map for the HDD.The block map may store at least two different types of information. Forexample, the block map may store information about locations at whichdata blocks on the HDD were intended to be stored and may also storeinformation about locations at which data blocks are actually stored onthe HDD. The information about where a block was intended to be storedmay include, for example, information about desired neighbors. Forexample, if a series of data blocks all came from a single file, thenthe block map may store an identifier of the immediately preceding blockfrom the single file, may store an identifier of the immediatelysucceeding block from the single file, and may store an identifier ofthe single file. This information may later facilitate relocating blocksfrom the file in an order that will yield more and longer sequentialreads.

Method 600 also includes, at 650, maintaining an access map for the HDD.The access map stores values associated with accesses (e.g., reads) todata blocks stored on the HDD. In one embodiment, maintaining the accessmap includes, upon determining that a target data block on the HDD hasbeen accessed (e.g., read), manipulating a value in the access map forthe target data block to indicate that the target data block hasexperienced another access. For example, each time a data block is read,a read count for that data block may be manipulated in the access map.Manipulating the read count for the data block may include, for example,incrementing the read count or otherwise increasing the read count.

While a raw read count provides useful information, disk performance maydepend more on what is currently happening rather than what happened along time ago. Thus, in one embodiment, maintaining the access mapincludes manipulating the values of data blocks in the access map toindicate that the target data block has experienced one or more feweraccesses. In one embodiment, manipulating the values in the access mapto indicate that the target data block has experienced one or more feweraccesses includes reducing (e.g., decrementing, decreasing) the values.For example, periodically, or in response to an event, the read countfor data blocks may be decremented. Data blocks that continue to receivereads will continue to have increasing read counts while data blocksthat are not receiving reads will eventually have zero read counts. Theread count may therefore store information about how hot or cold a datablock is, where a hot data block is one that is currently experiencing athreshold number of read counts and a cold data block is one that iscurrently experiencing less than the threshold number of read counts.With the information about which data blocks are currently experiencingthe most reads available, a garbage collection process for the HDD canbe used to improve read performance.

Therefore, method 600 also includes, at 660, selectively relocating aselected data block stored on the HDD as part of a garbage collectionprocess. The relocation is based, at least in part, on the block map andthe access map. Rather than simply appending a data block to the end ofthe log as is done conveniently, method 600 may make a moresophisticated selection for where to relocate a data block. Thus,selectively relocating the selected data block may include, for example,identifying a set of data blocks on the HDD that have an access valuewithin a range that matches the access value for the data block. Indifferent examples the range may be relative or discrete. For example,identification may be based on data blocks having the highest readcounts, data blocks having read counts above a threshold, or in otherways. In one embodiment, data blocks may be ranked based on read countand then binned into an equal number of bins.

Once the set of data blocks has been identified, then the selected datablock that has a similar read count or access value as the set of datablocks may be moved to a location on the HDD that is within a thresholddistance of the set of data blocks. The threshold distance may be, forexample, within a threshold number of cylinders or on the same cylinder.More generally, data blocks that have similar temperatures will berelocated to be positioned together.

In one embodiment, selectively relocating the selected data block mayinclude a second step. The second step may include identifying, withinthe set of data blocks, a subset of data blocks that were meant to bestored contiguously. The data blocks that were meant to be storedcontiguously may be identified from the organizational information inthe block map. The organizational information may include, for example,preceding data blocks, succeeding data blocks, or file origins. In thisembodiment, once data blocks that were meant to be stored contiguouslyare identified, method 600 may include moving at least one member of thesubset of data blocks so that at least two members of the subset arestored contiguously.

Different regions on an HDD may have different performancecharacteristics. For example, it may be more efficient in terms of timeor energy to perform i/o to the outermost cylinders of a HDD and it maybe less efficient in terms of time or energy to perform i/o to theinnermost cylinders of the HDD. The efficiency may be proportional tothe distance from the outermost cylinders on the HDD. Thus, in oneembodiment, selectively relocating the selected data block may includepositioning the selected data block with respect to the outermostcylinder on the HDD so that the distance of the selected data block fromthe outermost cylinder on the HDD is inversely proportional to the valuein the access map associated with the selected data block. Moregenerally, the hottest data blocks may be positioned on the outermostcylinders and the coldest data blocks may be positioned on the innermostcylinders.

FIG. 7 illustrates another embodiment of method 600. In this embodiment,method 600 includes, at 670, selectively relocating the selected datablock to a different HDD. For example, the data that is being read mostfrequently may be moved to a first HDD and the data that is being readleast frequently may be moved to a second different HDD. Computingefficiency and data storage efficiency may be improved by placing thedata that is read most frequently on a more efficient hard disk driveand by placing the data that is read least frequently on a lessefficient hard disk drive. Having disk drives with different capacities,efficiencies, energy costs, or other attributes, may increase theoverall utility of a data storage system and reduce the overall cost ofthe data storage system. Thus, method 600 may include not onlyrelocating data blocks on a single HDD but also migrating data blocksbetween different HDD.

In one example, a method may be implemented as computer executableinstructions. Thus, in one example, a computer-readable medium may storecomputer executable instructions that if executed by a machine (e.g.,processor) cause the machine to perform method 600. While executableinstructions associated with method 600 are described as being stored ona computer-readable medium, it is to be appreciated that executableinstructions associated with other example methods described herein mayalso be stored on a computer-readable medium.

FIG. 8 illustrates an apparatus 800 that improves HDD performance andefficiency using a log structured block device that performs selectivedata relocation. Apparatus 800 includes a processor 810, a memory 820,and a set 830 of logics that is connected to the processor 810 andmemory 820 by an interface 840. Memory 820 may store data about wheredata blocks were intended to be written on an HDD and where data blocksare actually written on the HDD. Memory 820 may also store data aboutaccesses (e.g., reads) of data blocks on the HDD. More generally, memory820 may store a heat map and an access map for the HDD.

In one embodiment, the functionality associated with the set of logics830 may be performed, at least in part, by hardware logic componentsincluding, but not limited to, field-programmable gate arrays (FPGAs),application specific integrated circuits (ASICs), application specificstandard products (ASSPs), system on a chip systems (SOCs), or complexprogrammable logic devices (CPLDs). In one embodiment, individualmembers of the set of logics 830 are implemented as ASICs or SOCs. Inone embodiment, the first logic 831, the second logic 832, or the thirdlogic 833 may be ASICs, FPGA, or other integrated circuits.

The set 830 of logics includes a first logic 831 that optimizes writesto a hard disk drive. The writes may be optimized by using a serializedlog structured block device approach that performs sequential writes.Data that is received may be serialized in the order in which it wasreceived irrespective of origin.

The apparatus 800 also includes a second logic 832 that maintains theblock map and the heat map for the hard disk drive. The block map storeslocation information about data blocks stored on the hard disk drive.The location information may be stored on a per block basis. Thelocation information may include, for example, a current location of adata block and a target location of a data block. The target locationmay be identified using organizational data. The organizational data mayinclude, for example, file origin information, desired neighborinformation, or other information.

The heat map stores access data about data blocks stored on the harddisk drive. The heat map may store read data on a per block basis. Theaccess data value is directly proportional to the number of times a datablock has been accessed (e.g., read) recently. The access data mayrecord, for example, the number of times a data block has been readwithin a relevant time frame. Thus, the second logic 832 may maintainthe heat map by incrementing a value for a data block each time it isread. While incrementing is described, other manipulations may beemployed. Since recent read activity may be more important to currentdisk performance than past read activity, the access data may bemanipulated to age out or otherwise de-emphasize older read activity.For example, the second logic 832 may periodically subtract a value fromthe access data for data blocks. In one embodiment, the access data forall data blocks may be decreased at the same time. The second logic 832may also subtract a value from the access data for data blocks inresponse to an event (e.g., threshold number of write occurring,threshold number of reads occurring). For example, if the disk is onlybeing written to, and there are very few reads, it may not be worthwhileto distinguish data blocks based on read frequency. In this case, theaccess data may only be manipulated after a certain number of reads haveoccurred, after the disk is more than a certain percent full, or uponthe occurrence of another event that indicates it may be worthwhile tostart tracking read activity.

The apparatus 800 also includes a third logic 833 that optimizes readsfor the hard disk drive by selectively relocating data on the hard diskdrive. The data blocks are relocated to increase a likelihood or lengthof a sequential read. Which data blocks are relocated is based, at leastin part, on the block map and the heat map. In one embodiment, the thirdlogic 833 selectively relocates data blocks having similar values intheir access data to similar regions of the hard disk drive. Forexample, data blocks having similar read counts may be moved to asimilar region of the hard disk drive. By way of illustration, datablocks that are in the top ten percent with respect to read counts maybe moved to a first set of cylinders while data blocks that in thebottom ten percent with respect to read counts may be moved to a secondset of cylinders. Data blocks that are neither in the top or bottom tenpercent may be considered uninteresting and not relocated. In oneembodiment, read counts may be binned into a finite number of binscorresponding to a number of regions on the hard disk drive. Data blocksmay then be relocated based on bin membership. In one embodiment, thesimilar regions of the hard disk drive are arranged in order from theouter cylinders of the hard disk drive to the inner cylinders of thehard disk drive, where the order is a function of the access count ofblocks in the region.

Once data blocks are located with other data blocks having similar readcounts, the data blocks may be further repositioned. In one embodiment,the third logic 833 selectively relocates data blocks within a region sothat data blocks having organizational data identifying that the datablocks are intended to be read sequentially are positioned to facilitatebeing read sequentially. For example, data blocks from a common filethat end up in a similar region may be positioned beside each otherbased on the heuristic that data from the same file may be readtogether.

More generally, the methods and apparatus described herein may be partof a system that includes a hard disk drive, a computer, and a datastore efficiency apparatus. The computer runs a log structured blockdevice process for the hard disk drive. The log structured block deviceprocess serializes data blocks to be written on the hard disk drive,causes the serialized data blocks to be written sequentially to the harddisk drive, and then stores data about the actual location for datablocks stored on the hard disk drive. In addition to information aboutwhere a data block was actually stored, the log structured block devicealso stores data about desired neighboring blocks for a data block. Asdata blocks are read, the process stores read counts for data blocks onthe hard disk drive. To identify more recent read activity, the processperiodically decrements read counts for data blocks on the hard diskdrive. With information about the read behavior being experienced bydata blocks on the hard disk drive available, the garbage collectionprocess for the hard disk drive can be enhanced to produce improvementsover conventional systems. Thus, the system includes a data storageefficiency apparatus that selectively relocates data blocks on the harddisk drive to store blocks with similar read counts within a thresholdnumber of cylinders on the hard disk drive and to store blocks that aredesired neighboring blocks on the same cylinder on the hard disk drive.More generally, data blocks that are experiencing similar read behaviormay be grouped together in a geographically related set of cylinders ortracks on the hard disk drive. Data blocks that end up in the samegeographic area may then be further relocated to increase the likelihoodof sequential reads by considering data including file origins, fileorders, or data block orders.

In one embodiment, a hard disk drive may be configured with anapplication specific integrated circuit (ASIC) or other integratedcircuit that is configured to provide a log structured block device forthe hard disk drive. The log structured block device serializes datablocks to be written on the hard disk drive, causes the serialized datablocks to be written sequentially to the hard disk drive, and thenstores data about the actual location for data blocks stored on the harddisk drive. In addition to information about where a data block wasactually stored, the log structured block device also stores data aboutdesired neighboring blocks for a data block. As data blocks are read,the process stores read counts for data blocks on the hard disk drive.To identify more recent read activity, the process periodicallydecrements read counts for data blocks on the hard disk drive. Withinformation about the read behavior being experienced by data blocks onthe hard disk drive available, the garbage collection process for thehard disk drive can be enhanced to produce improvements overconventional systems. Thus, the system includes a data storageefficiency apparatus that selectively relocates data blocks on the harddisk drive to store blocks with similar read counts within a thresholdnumber of cylinders on the hard disk drive and to store blocks that aredesired neighboring blocks on the same cylinder on the hard disk drive.More generally, data blocks that are experiencing similar read behaviormay be grouped together in a geographically related set of cylinders ortracks on the hard disk drive. Data blocks that end up in the samegeographic area may then be further relocated to increase the likelihoodof sequential reads by considering data including file origins, fileorders, temporal read behavior, or data block orders.

While example systems, methods, and other embodiments have beenillustrated by describing examples, and while the examples have beendescribed in considerable detail, it is not the intention of theapplicants to restrict or in any way limit the scope of the appendedclaims to such detail. It is, of course, not possible to describe everyconceivable combination of components or methodologies for purposes ofdescribing the systems, methods, and other embodiments described herein.Therefore, the invention is not limited to the specific details, therepresentative apparatus, and illustrative examples shown and described.Thus, this application is intended to embrace alterations,modifications, and variations that fall within the scope of the appendedclaims.

The following includes definitions of selected terms employed herein.The definitions include various examples and/or forms of components thatfall within the scope of a term and that may be used for implementation.The examples are not intended to be limiting. Both singular and pluralforms of terms may be within the definitions.

References to “one embodiment”, “an embodiment”, “one example”, “anexample”, and other similar terms, indicate that the embodiment(s) orexample(s) so described may include a particular feature, structure,characteristic, property, element, or limitation, but that not everyembodiment or example necessarily includes that particular feature,structure, characteristic, property, element or limitation. Furthermore,repeated use of the phrase “in one embodiment” does not necessarilyrefer to the same embodiment, though it may.

“Computer-readable storage medium”, as used herein, refers to anon-transitory medium that stores instructions and/or data. Acomputer-readable medium may take forms, including, but not limited to,non-volatile media, and volatile media. Non-volatile media may include,for example, optical disks, magnetic disks, and other disks. Volatilemedia may include, for example, semiconductor memories, dynamic memory,and other memories. Common forms of a computer-readable medium mayinclude, but are not limited to, a floppy disk, a flexible disk, a harddisk, a magnetic tape, other magnetic medium, an ASIC, a CD, otheroptical medium, a RAM, a ROM, a memory chip or card, a memory stick, andother media from which a computer, a processor or other electronicdevice can read.

“Data store”, as used herein, refers to a physical and/or logical entitythat can store data. A data store may be, for example, a database, atable, a file, a data structure (e.g. a list, a queue, a heap, a tree) amemory, a register, or other repository. In different examples, a datastore may reside in one logical and/or physical entity and/or may bedistributed between two or more logical and/or physical entities.

“Logic”, as used herein, includes but is not limited to hardware,firmware, software in execution on a machine, and/or combinations ofeach to perform a function(s) or an action(s), and/or to cause afunction or action from another logic, method, and/or system. Logic mayinclude, for example, a software controlled microprocessor, a discretelogic (e.g., ASIC), an analog circuit, a digital circuit, a programmedlogic device, or a memory device containing instructions. Logic mayinclude one or more gates, combinations of gates, or other circuitcomponents. Where multiple logical logics are described, it may bepossible to incorporate the multiple logical logics into one physicallogic. Similarly, where a single logical logic is described, it may bepossible to distribute that single logical logic between multiplephysical logics.

An “operable connection”, or a connection by which entities are“operably connected”, is one in which signals, physical communications,or logical communications may be sent or received. An operableconnection may include a physical interface, an electrical interface, ora data interface. An operable connection may include differingcombinations of interfaces or connections sufficient to allow operablecontrol. For example, two entities can be operably connected tocommunicate signals to each other directly or through one or moreintermediate entities (e.g., processor, operating system, logic,software). Logical or physical communication channels can be used tocreate an operable connection.

“Signal”, as used herein, includes but is not limited to, electricalsignals, optical signals, analog signals, digital signals, data,computer instructions, processor instructions, messages, a bit, or a bitstream, that can be received, transmitted and/or detected.

“Software”, as used herein, includes but is not limited to, one or moreexecutable instructions that cause a computer, processor, or otherelectronic device to perform functions, actions and/or behave in adesired manner. “Software” does not refer to stored instructions beingclaimed as stored instructions per se (e.g., a program listing). Theinstructions may be embodied in various forms including routines,algorithms, modules, methods, threads, or programs including separateapplications or code from dynamically linked libraries.

“User”, as used herein, includes but is not limited to one or morepersons, software, logics, applications, computers or other devices, orcombinations of these.

To the extent that the term “includes” or “including” is employed in thedetailed description or the claims, it is intended to be inclusive in amanner similar to the term “comprising” as that term is interpreted whenemployed as a transitional word in a claim.

To the extent that the term “or” is employed in the detailed descriptionor claims (e.g., A or B) it is intended to mean “A or B or both”. Whenthe applicants intend to indicate “only A or B but not both” then theterm “only A or B but not both” will be employed. Thus, use of the term“or” herein is the inclusive, and not the exclusive use. See, Bryan A.Garner, A Dictionary of Modern Legal Usage 624 (2d. Ed. 1995).

Although the subject matter has been described in language specific tostructural features or methodological acts, it is to be understood thatthe subject matter defined in the appended claims is not necessarilylimited to the specific features or acts described above. Rather, thespecific features and acts described above are disclosed as exampleforms of implementing the claims.

What is claimed is:
 1. A non-transitory computer-readable storage devicestoring computer-executable instructions that when executed by acomputer cause the computer to perform a method that provides a logstructured block device for a hard disk drive, the method comprising:receiving data to be stored on a hard disk drive (HDD), where the HDDhas at least one spinning platter, and where the spinning platter isconfigured with a plurality of concentric writeable cylinders;serializing the data to be stored into serialized data; writing theserialized data onto the HDD as a series of data blocks using asequential write operation; maintaining a block map for the HDD, wherethe block map includes information about locations at which data blockson the HDD were intended to be stored on the HDD and includesinformation about locations at which data blocks are actually stored onthe HDD; maintaining an access map that stores values associated withaccesses to data blocks stored on the HDD, where maintaining the accessmap includes, upon determining that a target data block on the HDD hasbeen accessed, manipulating a value in the access map for the targetdata block to indicate that the target data block has experiencedanother access; and selectively relocating a selected data block storedon the HDD as part of a garbage collection process based, at least inpart, on the block map and the access map.
 2. The non-transitorycomputer-readable storage device of claim 1, where selectivelyrelocating the selected data block includes identifying a set of datablocks on the HDD, where members of the set of data blocks have anaccess value within a range, and moving the selected data block to alocation on the HDD that is within a threshold distance of the set ofdata blocks.
 3. The non-transitory computer-readable storage device ofclaim 2, where selectively relocating the selected data block includesidentifying, within the set of data blocks, a subset of data blocks thatwere meant to be stored contiguously, and moving at least one member ofthe subset of data blocks so that at least two members of the subset arestored contiguously.
 4. The non-transitory computer-readable storagedevice of claim 1, where manipulating the value in the access map forthe target data block to indicate that the target data block hasexperienced another access includes incrementing the value or increasingthe value.
 5. The non-transitory computer-readable storage device ofclaim 1, where maintaining the access map includes, periodically or inresponse to an event, manipulating the values of data blocks in theaccess map to indicate that the target data block has experienced one ormore fewer accesses.
 6. The non-transitory computer-readable storagedevice of claim 5, where manipulating the values in the access map toindicate that the target data block has experienced one or more feweraccesses includes decrementing the values or decreasing the values. 7.The non-transitory computer-readable storage device of claim 1, whereselectively relocating the selected data block includes positioning theselected data block with respect to the outermost cylinder on the HDD sothat the distance of the selected data block from the outermost cylinderon the HDD is inversely proportional to the value in the access mapassociated with the selected data block.
 8. The non-transitorycomputer-readable device of claim 1, where selectively relocating theselected data block includes moving the selected data block to adifferent HDD.
 9. The non-transitory computer-readable storage device ofclaim 3, where the information about locations at which data blocks onthe HDD were intended to be stored includes actual block addresses onthe HDD or information identifying an intended immediately precedingdata block for an individual block or information identifying anintended immediately succeeding data block for the individual block. 10.The non-transitory computer-readable storage device of claim 9, whereserializing the data to store into serialized data includes arrangingthe data to store in the order in which the data to store was received.11. The non-transitory computer-readable storage device of claim 10,where serializing the data to store into serialized data includes movinga data block out of the order in which it was received and into an orderdetermined by the presence of a related data block.
 12. Thenon-transitory computer-readable storage device of claim 1, where theHDD is a shingled magnetic recording (SMR) device.
 13. An apparatus,comprising: a processor; a memory that stores a block map and a heatmap; a set of circuits; and an interface that connects the processor,the memory, and the set of circuits; the set of circuits comprising: afirst circuit configured to optimize writes to a hard disk drive using aserialized log structured block device approach that performs sequentialwrites; a second circuit configured to maintain the block map and theheat map for the hard disk drive, where the block map stores locationinformation about data blocks stored on the hard disk drive on a perblock basis, where the location information includes a current locationof a data block and a target location of a data block, and where theheat map stores access data about data blocks stored on the hard diskdrive on a per block basis, where the access data is directlyproportional to the number of times a data block has been accessed,where the second circuit is configured to maintain the heat map byadding a value to the access data for an accessed data block upondetermining that the accessed data block has been accessed; where thesecond circuit is configured to maintain the heat map by periodically orin response to an event, subtracting a value from the access data fordata blocks; and a third circuit configured to optimize reads for thehard disk drive by selectively relocating data on the hard disk drive toincrease a likelihood or length of a sequential read based, at least inpart, on the block map and the heat map.
 14. The apparatus of claim 13,where the second circuit maintains the block map by storing data aboutan actual location for a data block on the hard disk drive and bystoring organizational data about the data block.
 15. The apparatus ofclaim 13, where the third circuit is configured to selectively relocatedata blocks having similar values in their access data to similarregions of the hard disk drive.
 16. The apparatus of claim 15, where thethird circuit is configured to selectively relocate data blocks within aregion on the hard disk drive so that data blocks having organizationaldata identifying that the data blocks are intended to be readsequentially are positioned to facilitate being read sequentially. 17.The apparatus of claim 16, where the similar regions of the hard diskdrive are arranged in order from the outer cylinders of the hard diskdrive to the inner cylinders of the hard disk drive, where the order isa function of the access count of blocks in the region.